Dynamics of Scale Free Random Threshold Network 
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We study the dynamics of Random Threshold Network (RTN) on scale free networks, with asym- 
metric links, some interaction rules where propagation of local perturbations depends on in-degree 
k of the nodes. We find that there is no phase transition with respect to average connectivty inde- 
pendently of network topology for the case temperature T = 0, threshold h = and the probability 
distribution of indegree P{k) satisfies P{0) = D = 0. We have investigated the emergence of phase 
transition involving three parameters, i.e. T, h and D. RTN can be continuously connected to Ran- 
dom Boolean Network (RBN) in T —> cxa, and we find moderate thermal noise extends the regime 
of ordered dynamics, compared to RTN in T = regime and RBN. Furthermore, we discuss the 
dynamic properties from another point of view, dynamical mean field reaction rate equation. 

PACS numbers: PACS: 89.75.Fb, 89.20.Hh, 05.70.Fh, 87.23.Kg 



Recently there has been attracted considerable interest 
in the formation of random networks, in the connectivity 
and especially in the dynamics of these networks 0,0, Q^. 
Many naturally occuring networks such as the Internet 
II, WWW |5i, gene regulatory networkjQ], protein net- 
works 0, exhibit a power-law; i.e. scale free degree 
distribution for the links. A scale free network have a 
distinctive property that a small fraction of the nodes 
which is often called 'hub' are highly connected whereas 
the majority of the nodes have low connectivities. The 
absence of a typical scale of the connectivity is often re- 
lated to the organization of the network as a hierarchy. 
Although their topological properties have been studied 
in detail, dynamical behavior provided with asymmet- 
ric links and some interaction rules are not sufficiently 
explored. Especially in neural networks and gene regu- 
latory networks, interaction rules between nodes plays 
an important role to organize the dynamical proper- 
ties of the network. Such a dynamical network model, 
e.g. Random Boolean Network (RBN) is first introduced 
by Kauffman U 0| to model the gene regulatory ne- 
toworks in the biological system. In this models the state 
tJi S {0, 1} of a network node i is a logical function of the 
states of in-degree ki other nodes chosen at random. The 
logical functions are chosen at random, with a suitably 
biased probability. RBN model shows a phase transi- 
tion with respect to the number of inputs per node K 
at a critical average connectivity (Kc) = [2p(l — p)]^^ 
from ordered to chaotic state, where p is the probability 
that the randomly chosen output of node i are unity. In 
chaotic regime (K > Kc), a small perturbation in the 
initial state propagates across the entire system, whereas 
all perturbation in the initial state dies out in ordered 
regime (K < Kc). The dynamics of RBN on scale free is 
recently investigated [ll] |. However, its dynamical prop- 
erty is the same as that of classical RBN model because 
the output state of the nodes in RBN is independent of 
their in-degree distribution. In order to study the dynam- 
ical effects of the in-degree (scale free) distribution, we 



focus on Random Threshold Network (RTN), first inves- 
tigated as diluted and asymmetric spinglass models |l2l |. 
and asymmetric neural networks 13, 14] . It is a subset 
of Boolean Network which is thought to show dynamical 
behavior similar to RBN fl5]. 

We consider a network of N randomly interconnected 
binary nodes with state ct; = ±1. In this model, prob- 
ability distribution of outputs strongly depends on their 
in-degree parameter. At time t, the fields fi{t) of node i 
are computed: 



N 



Mt) = ^^Cijajit) 



(1) 



where h{> 0) is the threshold parameter. Threshold 
characteristics can be often shown in neural networks and 
gene regulatory networks, i.e. threshold for neuron firing 
and that for gene expression. In this brief report. The 
interaction weight Cy take discrete values Cij = ±1, with 
equal posiibility. If node i does not take a signal from j, 
one has Cy = 0. We assume cu is always zero to avoid 
self connection. The in-degree hi of node i is defined 



as ki = X] 



N 



The interaction weights Cij is chosen 



randomly from all nodes but follows the in-degree distri- 
bution P{k). For each node i, its state at time t -\- 1 are 
functions of the inputs it receives from other nodes at 
time t. 



Mt+^) = 



1, with probability G(/i(t)) 
T, with probability G{—fi{t)) 



(2) 



where G{x) = [1 + exp(— 2a;/r)]^^. The connection be- 
tween RBN and RTN is continuously made by introduc- 
ing thermal noise to probability distribution of outputs 
function G{x). The parameter T defines the temperature 
of the system. The N network nodes are idealized by 
synchronized updating. Generalizing so-called Anneal- 
ing Approximation method introduced by Derrida and 
Pomeau j3| , we approach the system analytically in the 
limit of a large system size N. One finds that normalized 



2 



overlap function x obey the equation 

x{t + \) = F{x{t)) (3) 
where the mapping function F{x) is given by 

K 

F{x) = l-Y,ps{k,h,T)P{k){l~x^) (4) 



is a subtraction of the normahzed Hamming distance 
from unity where Hamming distance d{a{t),a'{t)) de- 
notes overlap between time independent two distinctive 
configurations. In the N ^ oo limit, x represent the 
probability of two arbitary configurations to be equal. 
The possibility Ps is output state reversal by changing 
the state of a single input j. Using combinatorial meth- 
ods ^3 J the stochastic distribution of ps is derived. 



Normalized overlap function x{t) = 1 — d{a{t)^a'{t))/N 



Ps{k, h, T) = fcik + h-2i)(k-{k- i)G{k + h-2-2i)~ iG{k + h + 2- 2i)) 

+ {l-G{k + h- 2i)) {{k - i)G{k + h-2-2i) + iG{k + h + 2- 2i))^ 



(5) 



Stationary value of overlap x* is obtained by means of 
fixed point equation x — F(x). The existence of a phase 
transition depends uniquely on the nature of the fixed 
point X — I. If it is attractive, two initial configurations 
differing by an infinitesimal fraction of nodes will become 
almost identical, whereas if it is repulsive, they will pro- 
duce diverging trajectories. A possible critical point is 
then determined by the equation 



dx 



1 



(6) 



Let us recall briefly sufficient conditions for attractive 
fixed point x* , that is written as, Eq.Q satisfies 
{a)dF/dx > for all < X < 1, (b)F(O) > and 
{c)dF / dx\x=i > 1. For T ~ 0, ps is given by, 



Ps{k,h) 



,{k + h)/2 
Ps{k - 1, h) 



72* 



for k 
for k 



h is even 
h is odd 



(7) 



Moreover, h is also set to zero. Item (a), (b) are man- 
ifest, and (c) is surprisingly satisfied for any P{k) if 
P(0) = 0. It is straight-forward to prove (c), that we 
show kps{k) — 1 > 0,forVfc > 1, using asymptotically 
Ps{k) ~ k~^^^ for large k. In this case, fixed point x — 1 
is always repulsive and another fixed point x* < 1 is sta- 
ble. That is, independent of the in-degree distribution 
P(fc), there is no phase transition and a small pertur- 
bation/damage could propagate over the entire system. 
We now introduce the scale free in-degree distribution ki 
given by. 



P(fc) 



{l^D)r^{^,Kr^k- 



fc = 
l<k<k 



(8) 
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FIG. 1: The normalized overlap function x for t ^ oo as a 
function of 7 with h = 0(A), h = l(o), h — 2(x). The dashed 
line is that of RBN with p = 1/2. Cutoff K = 100 and 
temperature T — 0. The critical 7c doesn't exist (infinity) for 
ft = 0, 2.007 for h = l, 1.551 for h = 2; i.e., (K^) = l{h = 0), 
{Kc) = 3.140(/i = 1), (/Cc) = 6.800(/!, = 2). The inset shows 
time evolution of overlap function x{t) with a::(0) = 0.7 and 
7 — 2.5 for each threshold. 

where 77(7, K) = J^f with cutoff K, Q < D < I 
and 7 is usually called scale free exponent. Substituting 
Eq.(|HJl to Eq.®, we obtain the critical condition for D. 



Dr = 



Aj-f) - 1 
A(7) ' 



K 



A(7) = r;(7,/0-i^p,(fc)fci-^ (9) 



fe=i 



Since 1 ~ ps{k,0) > for V/c > 1, critical average con- 
nectivity (Kc) = 77(7 — 1, K)/{A{'y) * 77(7, K)) has a min- 
imum value in the 7 — *■ 00, i.e. minimum critical average 
(Kc) = 1 is realized when Dc = 0. 

In Fig^ we plot the stationary overlap function x* for 
various threshold parameters, provided that T = D = 0. 
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FIG. 2: Critical 7c curve as a function of T. In the chaotic 
(7 < 7c{T)) regime, overlap function in the stationary state 
x* satisfies x* < 1. In the ordered (7 > 7c(r)) regime, x* = 1. 
Threshold h = &nd cutoff K = 100. The inset (A') - T 
diagram shows {Kc} curve has maximum value aroud T = 
1.97. 

The overlap function for the case h — is shown to 
asymptotically approach x* ^ 1 with increasing 7, i.e. 
average connectivity is approaching (K) — 1. It is ob- 
viously different from RBN in which exists well known 
chaotic to ordered phase transition. For finite thresh- 
olds ft,, we observed equilibrium overlap is decreased as 
threshold incereases, which suggests they eliminates the 
memory of initial condition and two different initial con- 
dition become identical. For the comparison, The curve 
of equilibrium overlap for RBN is denoted. One observes 
a pertuabation is rather suppressed in RTN with zero- 
threshold than in RBN for low 7 regime. The opposite is 
observed for high 7 regime. The critical point where the 
strength of perturbation reverse is given by 7s ~ 1.754 
in this model. This result supports the numerical simu- 
lation observed in Fig. 8 of |l7| . 

Fig[21 shows the curve of critical scale free exponent 
7c as a function of temperature for ft = Z? = 0, which 
distinguish ordered state from chaotic state. For finite 
temperature T, dF/dx\x=i in large 7 limit is given by, 

d£ _ 1 

d^'"='- ~ (l + exp(-2/r))2 

+ (l-Hexp(2/T))2 +^(^/-^) 

The existence of infinitesimal T which satisfies 
dF/dx\x=i < 1 results in critical temperature Tc ~ 0. 
The critical line would diverge in the zero temperature 
limit. An interesting feature of the 7c curve is that 
it has a minimum value 7c = 2.1397 , i.e. maximum 
(Kc) =2.611, at T 1.97± .003. The same phenomenon 
can be observed in other connectivity models, for exam- 
ple, Poisson distribution model where the in-degree dis- 
tribution follows P{k) ~ e^^''Kp/kl. Parameter Kp is 
the average connectivity of the network. It has the crit- 
ical average connectivity (Kc) = 1.8494 at T = [l7j. 



In the low T regime, the higher the temperature T, the 
higher the critical average connectivity (Kc), and has a 
maximum value {K^} = 2.1994 around T = 1.683 ± .001 
(subsequently goes to {Kc) = 2). Both results show that 
moderate thermal noise can extend the regime of ordered 
dynamics in RTN to higher average network connectivi- 
ties, compared to the zero temperature and the infinite 
temperature regime. The infinite temprature T — > 00 
asymptotically leads to ps = 1/2, which is identical to 
RBN withp = 1/2. Substituting it to Eq.®, the critical 
condition in RBN is given by. 

Since average connectivity of the network is given by 
(K) = r]{-l +-/,K)/vil,K), one finds (Kc) = 2 from 
Ea. (|ll|) . Notice this value is independent of K and 7. 
Indeed, it is independent of any topological structures. 
In the K 00 limit, Ea. (|ll|) is represented by Riemann 
C function, which gives the critical value 7c ~ 2.4788. 

In the following, we approach the system from another 
point of view, by describing the single-site equation gov- 
erning the time evolution of the node state. We show a 
simple example of agent's strategy game. We first take 
the node state cri(t) as agent's strategy at time t, i.e. 
ai{t) = 1 as strategy A, and (Ji{t) = — 1 as strategy B. 
One can define Pk {t) as a density of agents which have 
connectivity k and take strategy B at time t. We assume 
the interaction weight Cy takes only 1 and and thresh- 
old ft to zero to simplify the dynamics. This alteration 
does not change the dynamical overlap equation Eq.ljSJ. 
This model can be also interpreted as a 'locally majority 
selection game' which means each agent choose the strat- 
egy at time t+1 the same as the majority of their inputs 
at time t. If the number of inputs strategies A and B is 
equal, the agent choose either one with equal probabil- 
ity. The dynamical mean field reaction rate equation for 
Pk{t) is written as [l^ll9j | 

dtPk{t) = -SkPkit) + iykil- Pk{t)) (12) 

Agents with connectivity k and strategy A would change 
its strategy to B with rate Vk, whereas agents with 
strategy B would change to A with rate 5k- By using 

p — P{k)pk which gives the average density of agents 
with strategy B, these rate parameters are written as 

i^k = 1-4 (13) 

Imposing the stationary condition on Ea. (|12|l . we find 
stationary densities = 1 — since pk is on its turn 
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FIG. 3: The stationary density of nodes with strategy A or 
B as a function of 7 for various T. K = 100. The density 
p is symmetry axis to p = 1/2. The numerical simulation 
have been run in networks of N = 10^ nodes and averaging 
over 50 different realizations. The full and dash line are the 
corresponding analytical results. Critical 7c which satisfies 
p(7) = 1/2 is 7c = 3.309(r = 0.5), 7c = 2.307(r = 1), 
7c = 1.976(r = 1.5), 

function of (5^ . A self consistency equation that allows us 
to find p* is given by: 

P - 1 - E E (^)p'(l - pf-'Gik - 2i)P{k) (14) 

k=l i=0 ^ ^ 

From Eg. 1141) . An asymptotic "f~T phase diagram similar 
to Fig|21is obtained. In 7 < jc{T) regime, the solution 
of Eq.lini) has 3 fixed points p = pi{< 1/2), 1/2, p2(> 
1/2). However, the fixed point at p = 1/2 is an unstable 
solution because the derivative of right hand in Ea. H14() 
has more than unity. Whether the density p goes to 
Pi or p2 in the stationary state depends on its initial 
condition. In 7 > ^c{T) regime, the equation has only 
one fixed and stable point at p = 1/2, which leads that 
any initial configurations asymptotically reach p = \/2 
in the stationary state. In Fig|21 we show the density p in 
the upper plane (p > 1/2) where the density p2 reaches 
/o = 1/2 as 7 increases. On the other hand, in the T — > 
limit, stationary density p is given by, 

lim pi — 0, lim po = l,for finite 7 (15) 

The time interval required to reach the equilibrium state 
t* is estimated as t* ~ . Since it takes long time to 
reach stationary state for large 7, the network would be 
practically fixed to its initial state for large 7 limit. In a 
small 7 area, the cutoff K become crucial, which leads to 
limT-.oo P2{l = 0) = k/{k + 1), limr^o ^2(7 = 0) = 1. 



For finite h case, p = is the only stable point for 7 > 7c, 
whereas there is another stable point for 7 < 7c. 

In conclusion, we have studied the properties of Ran- 
dom Threshold Network model on scale free networks. In 
h = Q,T — Q,D = Q case, we found there is no phase tran- 
sition with respect to average connectivity in any network 
topology. , i.e. propagation of perturbation/damage in- 
dependent of 7. Regarding to D, it is quite interesting 
that minimum critical average connectivity is obtained 
when the rate of fc = in-degree distribution is zero. We 
also clarifies the relationship between RTN and RBN, our 
findings that moderate thermal noise extends the regime 
of ordered dynamics to higher average connectivities, i.e. 
suppresses the propagation of perturbation/damage, sug- 
gest a new aspect to e.g. the theory of neural networks. 
Tuning thermal noise parameter in networks with asym- 
metric connections (as usually found in neural networks) 
where the basic units (neuron) follows threshold dynam- 
ics, thermal noise, i.e. non-zero error rate, may in fact 
improve information processing, given hierachical struc- 
ture of the network topology. 
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